This invention relates to the field of image-based vehicle occupant detection and classification. More specifically, the invention uses an imaging system in order to classify a vehicle seat into a number of occupancy classes, the minimum of which includes (i) empty, (ii) occupied by an adult passenger, (iii) occupied by a child passenger, (iv) occupied by a forward facing infant seat, (v) occupied by a rear facing infant seat.
Automobile occupant restraint systems that include an airbag are well known in the art, and exist in nearly all new vehicles being produced. While the introduction of passenger-side airbags proved successful in reducing the severity of injuries suffered in accidents, they have proven to be a safety liability in specific situations. Airbags typically deploy in excess of 200 mph and can cause serious, sometimes fatal, injuries to small or out-of-position occupants. These hazardous situations include the use of rear-facing infant seats (RFIS) in the front seat of a vehicle. While it is agreed upon that the safest location for a RFIS is the back seat, some vehicles do not have a back seat option. While RFIS occupants can be injured from indirect exposure to the force of an airbag, small children and occupants in forward-facing infant seats (FFIS) are at risk of injury from direct exposure to the airbag deployment. Beyond safety concerns, there is also a high financial cost associated with replacing a deployed airbag. Thus, it is preferred to deactivate an airbag when the passenger seat is empty, or occupied by an infant passenger.
Various solutions have been proposed to allow the modification of an airbag's deployment when a child or infant is occupying the front passenger seat. This could result in an airbag being deployed at a reduced speed, in an alternate direction, or not at all. The most basic airbag control systems include the use of a manual activation/deactivation switch controllable by the driver. Due to the nature of this device, proper usage could be cumbersome for the driver, especially on trips involving multiple stops. Weight sensors have also been proposed as a means of classifying occupants, but they may give inconsistent readings while an occupant is moving around in the seat. They may also be fooled by an over-cinched seat belt on an infant seat, and are prone to misclassification of heavy but inanimate objects. Capacitance-based sensors have also been proposed for occupant detection, but they have difficulty dealing with seat dampness.
Vision-based systems offer an alternative to weight-based and capacitance-based occupant detection systems. Intuitively we know that vision-based systems should be capable of detecting and classifying occupants, since humans can easily accomplish this task using visual senses alone. A number of vision-based occupant detection/classification systems have been proposed. In each of these systems one or more cameras are placed within the vehicle interior and capture images of the seat region. The seat region is then observed and the image is classified into one of several pre-defined classes such as “empty,” “occupied,” or “infant seat.” This occupancy classification can then act as an input to the airbag control system. Many of these systems, such as U.S. Pat. No. 5,531,472 to Steffens, rely on a stored visual representation of an empty passenger seat. This background template can then be subtracted from an observed image in order to generate a segmentation of the foreign objects (foreground) in the vehicle. This technique is highly problematic in that it relies on the system having a known image stored of the vehicle interior when empty, and will fail if cosmetic changes are made to the vehicle such as a reupholstery of the seat. As well, unless seat position and angle sensors are used (as suggested by Steffens), the system will not know which position the seat is in and will therefore have difficulty in extracting a segmented foreground image.
Other approaches include the generation of a set of image features which are then compared against a template reference set of image features in order to classify the image. This technique is used in U.S. Pat. No. 5,528,698 to Stevens, and U.S. Pat. No. 5,983,147 to Krumm, in both of which an image is classified as being “empty,” “occupied,” or having a “RFIS.” The reference set represents a training period that includes a variety of images within each occupant classification. However, generation of an exhaustive and complete reference set of image features can be difficult. As well, these systems are largely incapable of interpreting a scenario in which the camera's field-of-view is temporarily, or permanently, occluded. Some occupant detection systems have made use of range images derived from stereo cameras. Systems such as those in U.S. Pat. No. 5,983,147 to Krumm discuss the use of range images for this purpose, but ultimately these systems still face the challenges of generating a complete reference set, dealing with occlusion, and a means for segmenting the foreground objects. Finally, all of these systems that rely on a training set require that the classifier function be retrained if the camera mount location is moved, or used in a different vehicle.